Methods and systems for improving resource content mapping for an electronic learning system

ABSTRACT

Methods and systems for improving resource content mapping for an electronic learning system. The methods can include: receiving, by the electronic learning system, a resource for satisfying at least one learning objective of the one or more learning objectives, the resource comprising a content having a content data convertible into a text data and one or more resource property fields defining at least one characteristic of the resource; sectioning the content data into one or more content portions based on an analysis of at least one of the content data and the one or more resource property fields; and assigning at least one content portion of the one or more content portions to at least one learning objective.

CROSS-REFERENCE TO RELATED PATENT APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/729,612 filed on Jun. 3, 2015. The entire of U.S. patent applicationSer. No. 14/729,612 is hereby incorporated by reference.

TECHNICAL FIELD

The described embodiments relate to methods and systems associated withimproving resource content mapping for an electronic learning system.

INTRODUCTION

Electronic learning (also known as “e-Learning” or “eLearning”)generally refers to education or learning where users engage ineducation related activities using computers and other computingdevices. For example, users may enroll or participate in a course orprogram of study offered by an educational institution (e.g., a college,university or grade school) through a web interface that is accessibleover the Internet. Users may receive assignments electronically,participate in group work and projects by collaborating over theInternet, and be graded based on assignments and examinations that aresubmitted, for example, using an electronic submission tool.

Electronic learning is not limited to use by educational institutions.Electronic learning may be used in other environments, such asgovernment and corporations. For example, employees at a regional branchoffice of a corporation may use electronic learning to participate in atraining course offered by another office, or even a third-partyprovider. As a result, the employees at the regional branch office canparticipate in the training course without having to travel to the siteproviding the training course. Travel time and costs can be reduced andconserved.

In addition to enabling convenient access to electronic learning,electronic learning systems can collect data associated with its usageand adapt the electronic learning based on the collected data. Intraditional learning environments, a curriculum is typically static andrarely can the curriculum be adjusted based on individual needs. As aresult, teachers in traditional learning environments are unable todetermine the effectiveness of the curriculum and are limited in theirability to adapt the curriculum to the different needs of the learners.

SUMMARY OF SOME EMBODIMENTS

The various embodiments described herein generally relate to methods(and associated systems configured to implement the methods) forimproving resource content mapping for an electronic learning system.

In accordance with some embodiments, there is provided a method forimproving resource content mapping for an electronic learning system.The electronic learning system including a processor and at least onememory in electronic communication with the processor, the at least onememory storing one or more learning objectives. The method can include:receiving, by the electronic learning system, a resource for satisfyingat least one learning objective of the one or more learning objectives,the resource including a content having a content data convertible intoa text data and one or more resource property fields defining at leastone characteristic of the resource; sectioning the content data into oneor more content portions based on an analysis of at least one of thecontent data and the one or more resource property fields; and assigningat least one content portion of the one or more content portions to atleast one learning objective.

In some embodiments, sectioning the content data into the one or morecontent portions based on the analysis of the at least one of thecontent data and the one or more resource property fields includes:determining whether the one or more resource property fields includesone or more resource structure fields, the one or more resourcestructure fields defining a content structure of the content data andthe content structure including one or more data hierarchy levels; andin response to determining the one or more resource property fieldsincludes the one or more resource structure fields, sectioning thecontent data into the one or more content portions according to at leastone data hierarchy level of the one or more data hierarchy levels.

In some embodiments, assigning the at least one content portion to theat least one learning objective includes: applying a semantic analysisto the at least one content portion and applying the semantic analysisto each learning objective of the one or more learning objectives; basedon results of the semantic analysis, assigning a relevance score for theat least one content portion in respect of at least one learningobjective of the one or more learning objectives, the relevance scorerepresenting an estimated degree of correlation between the at least onecontent portion and the at least one learning objective; for eachlearning objective, determining whether the respective relevance scoreassigned to the at least one content portion at least satisfies arelevance threshold for that learning objective, the relevance thresholdbeing a minimum relevance score required for the at least one contentportion to be associated with that learning objective; and in responseto determining the relevance score at least satisfies the relevancethreshold, assigning the at least one content portion with that learningobjective.

In some embodiments, the one or more content portions includes two ormore data hierarchy levels.

In some embodiments, the resource includes a video, and receiving theresource for satisfying the at least one learning objective includestranscribing an audio data into the text data.

In some embodiments, the resource includes an image, and receiving theresource for satisfying the at least one learning objective includesapplying an electronic character recognition conversion to the image forgenerating the text data from the image.

In accordance with some embodiments, there is provided an electroniclearning system including: a memory for storing one or more learningobjectives; and a processor in electronic communication with the memory,the processor operating to: receive a resource for satisfying at leastone learning objective of the one or more learning objectives, theresource including a content having a content data convertible into atext data and one or more resource property fields defining at least onecharacteristic of the resource; section the content data into one ormore content portions based on an analysis of at least one of thecontent data and the one or more resource property fields; and assign atleast one content portion of the one or more content portions to atleast one learning objective.

In some embodiments, the processor operates to: determine whether theone or more resource property fields includes one or more resourcestructure fields, the one or more resource structure fields defining acontent structure of the content data and the content structureincluding one or more data hierarchy levels; and in response todetermining the one or more resource property fields includes the one ormore resource structure fields, section the content data into the one ormore content portions according to at least one data hierarchy level ofthe one or more data hierarchy levels.

In some embodiments, the processor operates to: apply a semanticanalysis to the at least one content portion and apply the semanticanalysis to each learning objective of the one or more learningobjectives; based on results of the semantic analysis, assign arelevance score for the at least one content portion in respect of atleast one learning objective of the one or more learning objectives, therelevance score representing an estimated degree of correlation betweenthe at least one content portion and the at least one learningobjective; for each learning objective, determine whether the respectiverelevance score assigned to the at least one content portion at leastsatisfies a relevance threshold for that learning objective, therelevance threshold being a minimum relevance score required for the atleast one content portion to be associated with that learning objective;and in response to determining the relevance score at least satisfiesthe relevance threshold, assign the at least one content portion withthat learning objective.

In some embodiments, the one or more content portions includes two ormore data hierarchy levels.

In some embodiments, the resource includes a video, and the processoroperates to transcribe an audio data into the text data.

In some embodiments, the resource includes an image, and the processoroperates to apply an electronic character recognition conversion to theimage for generating the text data from the image.

BRIEF DESCRIPTION OF THE DRAWINGS

Several embodiments will now be described in detail with reference tothe drawings, in which:

FIG. 1 is a schematic diagram of components interacting with anelectronic learning system in accordance with some embodiments;

FIG. 2 is a block diagram of some components that may be implemented inthe electronic learning system in accordance with an example embodiment;

FIG. 3 is a screenshot of an example user interface for receivingexample learning objectives by the electronic learning system inaccordance with an example embodiment;

FIG. 4 is a screenshot of an example user interface for receivingexample learning objectives by the electronic learning system inaccordance with another example embodiment;

FIG. 5 is a screenshot of an example user interface showing an examplelearning path generated based on the example learning objectivesreceived via the user interface in FIG. 3, in accordance with an exampleembodiment;

FIG. 6 is a screenshot of an example user interface showing an examplelearning path generated based on the example learning objectivesreceived via the user interface in FIG. 4, in accordance with an exampleembodiment;

FIG. 7 is a flowchart diagram of an example method for improvingresource content mapping for the electronic learning system, inaccordance with an example embodiment;

FIG. 8A is a flowchart diagram of an example method for providing alearning path for the electronic learning system, in accordance with anexample embodiment;

FIG. 8B is a flowchart diagram of an example method for updating aninitial learning path for the electronic learning system, in accordancewith an example embodiment;

FIG. 9 is a screenshot of an example user interface showing a modifiedversion of the learning path shown in FIG. 6, in accordance with anexample embodiment;

FIG. 10 is a flowchart diagram of an example method for modifying alearning path for the electronic learning system, in accordance with anexample embodiment;

FIG. 11 is a screenshot of an example user interface providing a portionof an example evaluation tool for receiving inputs from a user, inaccordance with an example embodiment;

FIG. 12 is a screenshot of an example user interface showing an updatedversion of the learning objectives received on FIG. 4 based on the userinputs received via the user interface in FIG. 11, in accordance withanother example embodiment;

FIG. 13 is a screenshot of an example user interface showing an examplelearning path generated based on the learning objectives in FIG. 12, inaccordance with an example embodiment;

FIG. 14 is a screenshot of an example user interface showing anotherexample learning path generated based on the learning objectives in FIG.12, in accordance with another example embodiment;

FIG. 15 is a flowchart diagram of an example method for providingevaluation resources for the electronic learning system, in accordancewith an example embodiment;

FIG. 16 is a screenshot of an example user interface showing anotherexample learning path generated based on the example learning objectivesreceived via the user interface in FIG. 4, in accordance with an exampleembodiment; and

FIG. 17 is a screenshot of an example user interface providing anevaluation resource in accordance with an example embodiment.

The drawings, described below, are provided for purposes ofillustration, and not of limitation, of the aspects and features ofvarious examples of embodiments described herein.

DESCRIPTION OF SOME EMBODIMENTS

For simplicity and clarity of illustration, elements shown in thedrawings have not necessarily been drawn to scale. The dimensions ofsome of the elements may be exaggerated relative to other elements forclarity. It will be appreciated that for simplicity and clarity ofillustration, where considered appropriate, reference numerals may berepeated among the drawings to indicate corresponding or analogouselements or steps. In addition, numerous specific details are set forthin order to provide a thorough understanding of the exemplaryembodiments described herein. However, it will be understood by those ofordinary skill in the art that the embodiments described herein may bepracticed without these specific details. In other instances, well-knownmethods, procedures and components have not been described in detail soas not to obscure the embodiments generally described herein.Furthermore, this description is not to be considered as limiting thescope of the embodiments described herein in any way, but rather asmerely describing the implementation of various embodiments asdescribed.

The embodiments of the systems and methods described herein may beimplemented in hardware or software, or a combination of both. In somecases, embodiments may be implemented in one or more computer programsexecuting on one or more programmable computing devices comprising atleast one processor, a data storage component (including volatile memoryor non-volatile memory or other data storage elements or a combinationthereof) and at least one communication interface.

For example and without limitation, the programmable computers (referredto below as computing devices) may be a server, network appliance,embedded device, computer expansion module, a personal computer, laptop,personal data assistant, cellular telephone, smart-phone device, tabletcomputer, a wireless device or any other computing device capable ofbeing configured to carry out the methods described herein.

In some embodiments, the communication interface may be a networkcommunication interface. In embodiments in which elements are combined,the communication interface may be a software communication interface,such as those for inter-process communication (IPC). In still otherembodiments, there may be a combination of communication interfacesimplemented as hardware, software, and combination thereof.

In some embodiments, each program may be implemented in a high levelprocedural or object-oriented programming and/or scripting language tocommunicate with a computer system. However, the programs can beimplemented in assembly or machine language, if desired. In any case,the language may be a compiled or interpreted language.

Program code may be applied to input data to perform the functionsdescribed herein and to generate output information. The outputinformation is applied to one or more output devices, in known fashion.

Each program may be implemented in a high level procedural or objectoriented programming and/or scripting language, or both, to communicatewith a computer system. However, the programs may be implemented inassembly or machine language, if desired. In any case, the language maybe a compiled or interpreted language. Each such computer program may bestored on a storage media or a device (e.g. ROM, magnetic disk, opticaldisc) readable by a general or special purpose programmable computer,for configuring and operating the computer when the storage media ordevice is read by the computer to perform the procedures describedherein.

In some embodiments, the systems and methods as described herein mayalso be implemented as a non-transitory computer-readable storage mediumconfigured with a computer program, wherein the storage medium soconfigured causes a computer to operate in a specific and predefinedmanner to perform at least some of the functions as described herein.

Furthermore, the systems, processes and methods of the describedembodiments are capable of being distributed in a computer programproduct comprising a computer readable medium that bears computer usableinstructions for one or more processors. The medium may be provided invarious forms, including one or more diskettes, compact disks, tapes,chips, wireline transmissions, satellite transmissions, internettransmission or downloadings, magnetic and electronic storage media,digital and analog signals, and the like. The computer useableinstructions may also be in various forms, including compiled andnon-compiled code.

Some of the embodiments described herein generally relate to methods(and associated systems configured to implement the methods) forproviding a learning path for a user of an electronic learning systemand in some embodiments, modifying the learning path for the user.

A learning path can include a series of one or more actions in respectof one or more resources that are accessible via the electronic learningsystem. The series of actions may be ordered in some embodiments. Aswill be described, the electronic learning system can define thelearning path based on a set of learning objectives, and in someembodiments, based also on one or more other factors. Generally, theelectronic learning system can operate to define the learning path sothat the learning path includes actions ordered in such a way to enablethe user to achieve the respective learning objectives.

A learning objective can be a desired achievement. The set of learningobjectives can be previously selected by the user, or assigned to agroup of individuals (including the user) by a third party (e.g., anemployer, a teacher, etc.).

The electronic learning system can collect and manage data associatedwith its usage. In traditional learning environments, such as aclassroom setting, teachers are generally unable to determine theeffectiveness of a curriculum while the course is underway. Also, acurriculum is typically quite static and therefore, teachers are rarelyequipped to adapt the curriculum to the different needs of the learners.

Unlike traditional learning environments, the electronic learningsystems described herein can provide the learning path for the user andadapt the learning path according to data associated with usage of thedescribed systems. Due, at least, to the speed at which the electroniclearning systems can operate and the amount of resources and/or datathat can be accessible to the electronic learning systems, theelectronic learning systems can relatively quickly, or possibly evennearly in real-time, customize the learning path for the relevant usersin response to the collected usage data.

Learning paths can be generated based on an estimated degree ofcorrelation between the learning objectives and the one or moreresources. In some embodiments, the electronic learning systemsdescribed herein can update the learning path based on usage dataassociated with certain types of resources, such as evaluationresources. An evaluation resource is a resource that involves somedegree of interaction between the user and the electronic learningsystem in order to evaluate a proficiency of the user with one or morelearning objectives. When a usage amount of the evaluation resource atleast satisfies a predefined threshold, the systems described herein candetermine that the collected usage data is sufficient and can thenproceed to update the learning path based, at least partially, on thatcollected usage data.

The electronic learning systems described herein can also modify thelearning path initially defined for the user based on response inputsreceived from the user. Each response input can be associated with atleast one learning objective. On receipt of the response inputs, theelectronic learning system can evaluate the received response inputs anddetermine whether the user is proficient with the relevant one or morelearning objectives.

In some of the described embodiments, evaluation resources can bedynamically generated in response to triggering actions conducted by theuser in respect of the learning path. The electronic learning systemscan customize the evaluation resources based on the user's priorinteraction with the learning path and/or usage data from other users.The dynamically generated evaluation resources can improve the user'sinteraction with the learning path. The electronic learning system mayalso adjust the remainder of the learning path to accommodate the user'sprogress in the learning path.

Some of the embodiments described herein generally relate to methods(and associated systems configured to implement the methods) forimproving resource content mapping for the electronic learning system.Resources can typically include large amount of contents that may berelevant to multiple different learning objectives, or in some cases,only a small portion of the contents in a resource is relevant to alearning objective. Certain resources that contain content convertibleto text format can be sectioned into content portions according to thedescribed methods, and accordingly assigned to the various one or morelearning objectives.

Referring now to FIG. 1, illustrated therein is a schematic diagram 10of components interacting with an electronic learning system 30 forproviding electronic learning according to some embodiments.

As shown in the schematic diagram 10, one or more users 12, 14 mayaccess the electronic learning system 30 to participate in, create, andconsume electronic learning services, including educational content suchas courses. In some cases, the electronic learning system 30 may be partof (or associated with) a traditional “bricks and mortar” educationalinstitution (e.g. a grade school, university or college), another entitythat provides educational services (e.g. an online university, a companythat specializes in offering training courses, an organization that hasa training department, etc.), or may be an independent service provider(e.g. for providing individual electronic learning).

It should be understood that a course is not limited to formal coursesoffered by formal educational institutions. The course may include anyform of learning instruction offered by an entity of any type. Forexample, the course may be a training seminar at a company for a groupof employees or a professional certification program (e.g. ProjectManagement Professional™ (PMP), Certified Management Accountants (CMA),etc.) with a number of intended participants.

In some embodiments, one or more educational groups 16 can be defined toinclude one or more users 12, 14. For example, as shown in FIG. 1, theusers 12, 14 may be grouped together in the educational group 16. Theeducational group 16 can be associated with a particular course (e.g.History 101 or French 254, etc.), for example. The educational group 16can include different types of users. A first user 12 can be responsiblefor organizing and/or teaching the course (e.g. developing lectures,preparing assignments, creating educational content, etc.), such as aninstructor or a course moderator. The other users 14 can be consumers ofthe course content, such as students.

In some examples, the users 12, 14 may be associated with more than oneeducational group 16 (e.g. some users 14 may be enrolled in more thanone course, another example user 12 may be a student enrolled in onecourse and an instructor responsible for teaching another course, afurther example user 12 may be responsible for teaching several courses,and so on).

In some examples, educational sub-groups 18 may also be formed. Forexample, the users 14 shown in FIG. 1 form an educational sub-group 18.The educational sub-group 18 may be formed in relation to a particularproject or assignment (e.g. educational sub-group 18 may be a lab group)or based on other criteria. In some embodiments, due to the nature ofelectronic learning, the users 14 in a particular educational sub-group18 may not need to meet in person, but may collaborate together usingvarious tools provided by the electronic learning system 30.

In some embodiments, other educational groups 16 and educationalsub-groups 18 could include users 14 that share common interests (e.g.interests in a particular sport), that participate in common activities(e.g. users that are members of a choir or a club), and/or have similarattributes (e.g. users that are male, users under twenty-one years ofage, etc.).

Communication between the users 12, 14 and the electronic learningsystem 30 can occur either directly or indirectly using any one or moresuitable computing devices. For example, the user 12 may use a computingdevice 20 having one or more device processors such as a desktopcomputer that has at least one input device (e.g. a keyboard and amouse) and at least one output device (e.g. a display screen andspeakers).

The computing device 20 can generally be any suitable device forfacilitating communication between the users 12, 14 and the electroniclearning system 30. For example, the computing device 20 could bewirelessly coupled to an access point 22 (e.g. a wireless router, acellular communications tower, etc.), such as a laptop 20 a, awirelessly enabled personal data assistant (PDA) or smart phone 20 b, atablet computer 20 d, or a game console 20 e. The computing device 20could be coupled to the access point 22 over a wired connection 23, suchas a computer terminal 20 c.

The computing devices 20 may communicate with the electronic learningsystem 30 via any suitable communication channels.

The computing devices 20 may be any networked device operable to connectto the network 28. A networked device is a device capable ofcommunicating with other devices through a network such as the network28. A network device may couple to the network 28 through a wired orwireless connection.

As noted, these computing devices may include at least a processor andmemory, and may be an electronic tablet device, a personal computer,workstation, server, portable computer, mobile device, personal digitalassistant, laptop, smart phone, WAP phone, an interactive television,video display terminals, gaming consoles, and portable electronicdevices or any combination of these. These computing devices may behandheld and/or wearable by the user.

In some embodiments, these computing devices may be a laptop 20 a, or asmartphone device 20 b equipped with a network adapter for connecting tothe Internet. In some embodiments, the connection request initiated fromthe computing devices 20 a, 20 b may be initiated from a web browser anddirected at the browser-based communications application on theelectronic learning system 30.

For example, the computing devices 20 may communicate with theelectronic learning system 30 via the network 28. The network 28 mayinclude a local area network (LAN) (e.g., an intranet) and/or anexternal network (e.g., the Internet). For example, the computingdevices 20 may access the network 28 by using a browser applicationprovided on the computing device 20 to access one or more web pagespresented over the Internet via a data connection 27.

The network 28 may be any network capable of carrying data, includingthe Internet, Ethernet, plain old telephone service (POTS) line, publicswitch telephone network (PSTN), integrated services digital network(ISDN), digital subscriber line (DSL), coaxial cable, fiber optics,satellite, mobile, wireless (e.g. Wi-Fi, WiMAX), SS7 signaling network,fixed line, local area network, wide area network, and others, includingany combination of these, capable of interfacing with, and enablingcommunication between the computing devices 20 and the electroniclearning system 30, for example.

In some examples, the electronic learning system 30 may authenticate anidentity of one or more of the users 12, 14 prior to granting the user12, 14 access to the electronic learning system 30. For example, theelectronic learning system 30 may require the users 12, 14 to provideidentifying information (e.g., a login name and/or a password) in orderto gain access to the electronic learning system 30.

In some examples, the electronic learning system 30 may allow certainusers 12, 14, such as guest users, access to the electronic learningsystem 30 without requiring authentication information to be provided bythose guest users. Such guest users may be provided with limited access,such as the ability to review one or more components of the course todecide whether they would like to participate in the course but withoutthe ability to post comments or upload electronic files.

In some embodiments, the electronic learning system 30 may communicatewith the access point 22 via a data connection 25 established over theLAN. Alternatively, the electronic learning system 30 may communicatewith the access point 22 via the Internet or another external datacommunications network. For example, one user 14 may use the laptop 20 ato browse to a webpage (e.g. a course page) that displays elements ofthe electronic learning system 30.

The electronic learning system 30 can include one or more components forproviding electronic learning services. It will be understood that insome embodiments, each of the one or more components may be combinedinto fewer number of components or may be separated into furthercomponents. Furthermore, the one or more components in the electroniclearning system 30 may be implemented in software or hardware, or acombination of software and hardware.

For example, the electronic learning system 30 can include one or moreprocessing components, such as computing servers 32. Each computingserver 32 can include one or more processor. The processors provided atthe computing servers 32 can be referred to as “system processors” whileprocessors provided at computing devices 20 can be referred to as“device processors”. The computing servers 32 may be a computing device20 (e.g. a laptop or personal computer).

It will be understood that although two computing servers 32 are shownin FIG. 1, one or more than two computing servers 32 may be provided.The computing servers 32 may be located locally together, or distributedover a wide geographic area and connected via the network 28.

The system processors may be configured to control the operation of theelectronic learning system 30. The system processors can initiate andmanage the operations of each of the other components in the electroniclearning system 30. The system processor may also determine, based onreceived data, stored data and/or user preferences, how the electroniclearning system 30 may generally operate.

The system processor may be any suitable processors, controllers ordigital signal processors that can provide sufficient processing powerdepending on the configuration, purposes and requirements of theelectronic learning system 30. In some embodiments, the system processorcan include more than one processor with each processor being configuredto perform different dedicated tasks.

In some embodiments, the computing servers 32 can transmit data (e.g.electronic files such as web pages) over the network 28 to the computingdevices 20. The data may include electronic files, such as webpages withcourse information, associated with the electronic learning system 30.Once the data is received at the computing devices 20, the deviceprocessors can operate to display the received data.

The electronic learning system 30 may also include one or more datastorage components 34 that are in electronic communication with thecomputing servers 32. The data storage components 34 can include RAM,ROM, one or more hard drives, one or more flash drives or some othersuitable data storage elements such as disk drives, etc. The datastorage components 34 may include one or more databases, such as arelational database (e.g., a SQL database), for example.

The data storage components 34 can store various data associated withthe operation of the electronic learning system 30. For example, coursedata 35, such as data related to a course's framework, educationalcontent, and/or records of assessments, may be stored at the datastorage components 34. The data storage components 34 may also storeuser data, which includes information associated with the users 12, 14.The user data may include a user profile for each user 12, 14, forexample. The user profile may include personal information (e.g., name,gender, age, birthdate, contact information, interests, hobbies, etc.),authentication information to the electronic learning system 30 (e.g.,login identifier and password) and educational information (e.g., whichcourses that user is enrolled in, the user type, course contentpreferences, etc.). The data storage components 34 may also store dataassociated with the learning path, such as learning objectives andlearning path data associated with the learning path.

The data storage components 34 can store authorization criteria thatdefine the actions that may be taken by certain users 12, 14 withrespect to the various educational contents provided by the electroniclearning system 30. The authorization criteria can define differentsecurity levels for different user types. For example, there can be asecurity level for an instructing user who is responsible for developingan educational course, teaching it, and assessing work product from thestudent users for that course. The security level for those instructingusers, therefore, can include, at least, full editing permissions toassociated course content and access to various components forevaluating the students in the relevant courses.

In some embodiments, some of the authorization criteria may bepre-defined. For example, the authorization criteria can be defined byadministrators so that the authorization criteria are consistent for theelectronic learning system 30, as a whole. In some further embodiments,the electronic learning system 30 may allow certain users, such asinstructors, to vary the pre-defined authorization criteria for certaincourse contents.

The electronic learning system 30 can also include one or more backupservers 31. The backup server can store a duplicate of some or all ofthe data 35 stored on the data storage components 34. The backup server31 may be desirable for disaster recovery (e.g. to prevent data loss inthe event of an event such as a fire, flooding, or theft). It should beunderstood that although only one backup server 31 is shown in FIG. 1,one or more backup servers 31 may be provided in the electronic learningsystem 30. The one or more backup servers 31 can also be provided at thesame geographical location as the electronic learning system 30, or oneor more different geographical locations.

The electronic learning system 30 can include other components forproviding the electronic learning services. For example, the electroniclearning system 30 can include a management component that allows users12, 14 to add and/or drop courses and a communication component thatenables communication between the users 12, 14 (e.g., a chat software,etc.). The communication component may also enable the electroniclearning system 30 to benefit from tools provided by third-partyvendors. Other example components will be described with reference toFIG. 2.

Referring now to FIG. 2, which is a block diagram 100 of some componentsthat may be implemented in the electronic learning system 30 accordingto some embodiments. In the example of FIG. 2, the various illustratedcomponents are provided at one of the computing servers 32.

As shown in FIG. 2, the computing server 32 may include a systemprocessor 110, a resource mapping component 112, a learning pathcomponent 114, an interface component 116, an evaluation component 118,a local storage component 120 and a data storage component 134.

Each of the system processor 110, the resource mapping component 112,the learning path component 114, the interface component 116, theevaluation component 118, the local storage component 120 and the datastorage component 134 can be in electronic communication with oneanother. It should be noted that in some embodiments, the systemprocessor 110, the resource mapping component 112, the learning pathcomponent 114, the interface component 116, the evaluation component118, the local storage component 120 and the data storage component 134may be combined or may be separated into further components.Furthermore, the system processor 110, the resource mapping component112, the learning path component 114, the interface component 116, theevaluation component 118, the local storage component 120 and the datastorage component 134 may be implemented using software, hardware or acombination of both software and hardware.

Generally, the system processor 110 controls the operation of thecomputing server 32 and, as a result, various operations of theelectronic learning system 30. For example, the system processor 110 mayinitiate the resource mapping component 112 for improving theassociation of resources to the learning objectives stored in the datastorage component 134. In some embodiments, the system processor 110 maybe configured to initiate the learning path component 114 to generate alearning path for a user and/or adapt the learning path in accordancewith the methods described herein. The system processor 110 may also,based on user response inputs received via the interface component 116,initiate the evaluation component 118 to determine a competence level ofthe user in respect of at least one learning objective, as will bedescribed. It will be understood that the system processor 110 is notlimited to the described operations and that the described operationsare provided only for the purpose of illustration.

The interface component 116 may be any interface that enables thecomputing server 32 to communicate with the other computing servers 32,backup servers 31 and data storage components 34 within the electroniclearning system 30. The interface component 116 may also include anyinterface that enables the computing server 32 to communicate withthird-party systems. In some embodiments, the interface component 116can include at least one of a serial port, a parallel port or a USBport. The interface component 116 may also include at least one of anInternet, Local Area Network (LAN), Ethernet, Firewire, modem or digitalsubscriber line connection. Various combinations of these elements maybe incorporated within the interface component 116.

In some embodiments, the interface component 116 may receive input fromthe computing devices 20 via various input components, such as a mouse,a keyboard, a touch screen, a thumbwheel, a track-pad, a track-ball, acard-reader, voice recognition software and the like depending on therequirements and implementation of the electronic learning system 30.

The local storage component 120 may be provided at the computing server32 for temporary storage of data associated with various operations ofthe system processor 110. The local storage component 120 may receivedata from and/or transmit data to the data storage component 134.

The data storage component 134 can include one or more databases. Forexample, as shown in FIG. 2, the data storage component 134 can includea learning objectives database 140, a resources database 142, a userdatabase 144, and a learning path database 146. Although each of thedatabases 140, 142, 144 and 146 are shown to be separate databases, itwill be understood that two or more of the databases 140, 142, 144 and146 may be provided together as fewer databases, or one or more of thedatabases 140, 142, 144 and 146 may be further separated into a greaternumber of databases.

The learning objectives database 140 can store data associated with thelearning objectives. In some embodiments, the electronic learning system30 can store one or more predefined learning objectives in the learningobjectives database 140. To facilitate the process of defining thelearning objectives, the electronic learning system 30 can provide oneor more of the predefined learning objectives from which the relevantlearning objectives can be selected. In some embodiments, the learningobjectives database 140 can store the learning objectives received andassigned to a learner.

Reference is now made to FIG. 3, which is a screenshot 200 of an exampleuser interface 210 for receiving example learning objectives 220, 230 bythe electronic learning system 30. The user interface 210 is providedvia a browser application 202 in this example.

As shown in FIG. 3, learning objectives can have different levels ofbreadth. Broader learning objectives, such as overall learningobjectives 220, can be directed to an achievement that can be applied toother subjects. Subject specific learning objectives 230 are more narrowlearning objectives than the overall learning objectives 220. In someembodiments, achieving one or more subject specific learning objectives230 can lead to the achievement of an overall learning objective 220. Itwill be understood that learning objectives may not be divided intodifferent levels of breadth, in some embodiments.

In FIG. 3, the overall learning objectives 220 include three differentoverall learning objectives 220, namely 220 a, 220 b and 220 c. Each ofthe overall learning objectives 220 a, 220 b and 220 c is directed at adifferent desired skill to be achieved by the learner, namely “Skill I”,“Skill II” and “Skill III”, respectively. The subject specific learningobjectives 230 include four different subject specific learningobjectives, namely 230 a, 230 b, 230 c and 230 d. The first subjectspecific learning objective 230 a requires that the learner understandsa specific topic, namely “Topic I”; the second subject specific learningobjective 230 b requires that the learner recognizes the relevance of“Topics II and III” to “Topic I”; the third subject specific learningobjective 230 c requires that the learner be able to identify featuresof another subject, “Topic V”; and the fourth subject specific learningobjective 230 d requires that the learner be able to relate “Topic V” to“Topic I”. It will be understood that the described learning objectives220, 230 are merely for illustrative purposes and learning objectivesare not limited in any way to the learning objectives 220, 230 shown inFIG. 3.

FIG. 4 is a screenshot 300 of another example user interface 310 forreceiving example learning objectives 320, 330 by the electroniclearning system 30. The user interface 310 is provided via a browserapplication 302 in this example. Similar to the example learningobjectives 220, 230 shown in FIG. 3, the learning objectives 320, 330 inFIG. 4 also include overall learning objectives 320 and subject specificlearning objectives 330.

The learning objectives 320, 330 shown in FIG. 4 are directed todeveloping an understanding of the fundamentals of marketing. Theoverall learning objectives 320 include a first overall learningobjective 320 a (the learner is required to be able to explain thestages of product development), and a second overall learning objective320 b (the learner is able to demonstrate an understanding of thestrategies in the promotion of goods, services and events). The subjectspecific learning objectives 330 include, at least, a first subjectspecific learning objective 330 a (the learner is required to be able toclassify products according to type), a second subject specific learningobjective 330 b (the learner is required to be able to identifycomponents of the product life cycle), a third subject specific learningobjective 330 c (the learner is required to describe factors that affectprice of a product), and a fourth subject specific learning objective330 d (the learner is required to identify channels of distribution).For ease of exposition, the other subject specific learning objectives330 are not shown in FIG. 4.

From FIG. 4, it can be seen that one or more of the subject specificlearning objectives 330 is relevant to at least one of the overalllearning objectives 320. For example, the subject specific learningobjectives 330 associated with products, such as 330 a and 330 b, arerelevant to the first overall learning objectives 320 a, while thesubject specific learning objective 330 associated with the strategiesin the promotion of goods, such as 330 c, is relevant to the secondoverall learning objective 320 b.

Returning to FIG. 2, the resources database 142 can store the resources,and associated resource data, that can be accessible by the electroniclearning system 30 for inclusion into the learning path by the user.Generally, various types of resources can be provided by the electroniclearning system 30. For example, the resources can include text data,video data, image data, and one or more combinations thereof.

In some embodiments, the resource can include content with content datathat is convertible into a text data format. The resource can alsoinclude one or more resource property fields that define at least onecharacteristic of the resource. For example, a resource property fieldcan indicate a content structure of the content data. The contentstructure can indicate a number of data hierarchy levels that may bepresent in the content. As will be described, the resource propertyfields can be used by the resource mapping component 112 for improvingthe mapping of the resources with the learning objectives stored in thelearning objectives database 140.

The user database 144 can store user data for the various users of theelectronic learning system 30. For example, the user data can include auser profile for each user 12, 14. The user profile may include personalinformation (e.g., name, gender, age, birthdate, contact information,interests, hobbies, etc.), authentication information to the electroniclearning system 30 (e.g., login identifier and password) and educationalinformation (e.g., which courses that user is enrolled in, the usertype, course content preferences, etc.). It will be understood that thedescribed user data is merely for illustrative purposes and that theuser data stored in the user database 144 is not limited to thedescribed examples.

The learning path database 146 can store path data associated with thelearning paths defined by the learning path component 114. For example,the path data can include identifiers corresponding to the resourcesselected to be included into the learning path based on the receivedlearning objectives, user identifiers corresponding to the users withaccess to the learning path (e.g., learners who follow the learning pathand teachers who define the learning path, etc.), and/or usage dataassociated with the interaction with the learning path by the respectiveusers. The learning path component 114 can include the software and dataassociated with the various methods for providing the learning path fora user and/or modifying the learning path, as described herein. Exampleembodiments will now be described with reference to, at least, FIGS. 8Ato 14.

Reference will now be made to FIG. 5. FIG. 5 is a screenshot of anexample user interface 400 showing an example learning path generatedbased on the example learning objectives 220, 230 received via the userinterface 210 in FIG. 3.

When the learning objectives 220, 230 are received by the electroniclearning system 30, the system processor 110 can initiate the learningpath component 114 to generate a learning path, such as learning path412 shown in FIG. 5. The learning path component 114 can generate thelearning path 412 based on the learning objectives 220, 230 received viathe user interface 210 by identifying, from the learning objectivesdatabase 140 and the resources database 142, the resources associatedwith the learning objectives 220, 230.

As generally shown in FIG. 5, the learning path 412 includes two groupsof actions, namely a first series of actions 430 associated with a firstgroup 420 a (“Topic I”) and a second series of actions 450 associatedwith a second group 420 b (“Topic II”). Each of the first and secondsets of actions 430 and 450 includes different actions in respect ofvarious different resources. In the first series of actions 430, thelearning path component 114 has included an action 432 to read theintroduction, an action 434 to read Article #1, an action 436 tocomplete Assignment #1, an action 438 to read Article #2, an action 440to watch a video on Topic I and an action 442 to complete an Assessment#1. In the second series of actions 450, the learning path component 114has included an action 452 to read Article #3, an action 454 to completeAssignment #2, an action 456 to participate in the Discussion Forum #1,an action 458 to read Article #4, an action 460 to watch a video onTopics I and V, and an action 462 to complete Assessment #2.

Reference will now be made to FIG. 6. FIG. 6 is a screenshot of anexample user interface 500 showing an example learning path generatedbased on the example learning objectives 320, 330 received via the userinterface 310 in FIG. 4. The learning path component 114 can generatethe learning path 512 based on the learning objectives 320, 330 receivedvia the user interface 310 by identifying, from the learning objectivesdatabase 140 and the resources database 142, the resources associatedwith the learning objectives 320, 330.

As generally shown in FIG. 6, the learning path 512 includes threegroups of actions, namely a first series of actions 530 associated witha first group 520 a (“Marketing Basics”), a second series of actions 540associated with a second group 520 b (“Products”) and a third series ofactions 560 associated with a third group 520 c (“Product & Pricing”).Each of the first, second and third sets of actions 530, 540 and 560includes actions in respect of various different resources.

In the first series of actions 530, the learning path component 114 hasincluded an action 532 to read an introduction article, an action 534 tocomplete Assignment I and an action 536 to complete Assessment I. In thesecond series of actions 540, the learning path component 114 hasincluded an action 542 to read an article regarding product types, anaction 544 to read an article regarding life cycle, an action 546 to usean interaction component, an action 548 to use another interactioncomponent, and an action 550 to complete Assessment II. In the thirdseries of actions 560, the learning path component 114 has included anaction 562 to read an article regarding pricing of products, an action564 to watch a video regarding pricing, an action to read an articleregarding the effects of distribution channels on pricing 566 and anaction 568 to complete Assessment III.

Referring again to FIG. 2, the resource mapping component 112 caninclude the software and data associated with the various methods forimproving resource content mapping for the electronic learning system30, as described herein. For example, when a resource is received viathe interface component 116, the system processor 110 can initiateoperation of the resource mapping component 112 to map the receivedresource. Example embodiments will be described with reference to FIG.7.

The evaluation component 118 can be operated by the system processor 110to determine a competence level of the user in respect of at least onelearning objective. For example, the evaluation component 118 cangenerate the dynamic evaluation resources as generated with the methodsand systems described herein. In some embodiments, the evaluationcomponent 118 and the learning path component 114 can operate togetherso that the learning path component 114 can modify the learning pathbased, at least, on the determinations by the evaluation component 118in respect of the competence level of the user. It will be understoodthat, in some embodiments, the evaluation component 118 may be providedas part of the learning path component 114. Example embodiments will bedescribed with reference to, at least, FIGS. 10 to 17.

Referring now to FIG. 7, a flowchart diagram illustrating an examplemethod 600 for improving resource content mapping for the electroniclearning system 30 is shown.

As described, the resources database 142 can store various differentresources to be accessible by the electronic learning system 30 forinclusion into the learning paths provided by the learning pathcomponent 114. The scope and degree of relevance of each resource canvary. For example, a resource can include content data that may berelevant to more than one learning objective, while in some other cases,only a small amount of a resource (e.g., one content portion) isrelevant to a learning objective. In order to improve the correlationbetween the learning objective and the resource, and to facilitateaccess to the relevant portions of the resource, the resource mappingcomponent 112 can process each resource according to the methodsdescribed with respect to FIG. 7.

At 610, the electronic learning system 30 receives a resource forsatisfying at least one learning objective of the one or more learningobjectives.

The electronic learning system 30 can receive one or more resources froma third party (e.g., a content provider, etc.) and/or a user (e.g.,learner, teacher, etc.) so that the resource can be included in thelearning paths provided by the learning path component 114 if theresource is determined to be relevant to one or more learning objectivesstored in the learning objectives database 140.

The resource can include content with content data that is convertibleinto a text data format, such as an article (e.g., a Wikipedia™ article)or an electronic book (“e-book”). In some embodiments, the resource caninclude video data and/or image data. When the received resourceincludes video data, the system processor 110 can determine whether theaudio data associated with the video data is available or can betranscribed into text data. When the received resource includes imagedata, the system processor 110 can determine whether any portion of theimage data can be converted into text data. For example, the systemprocessor 110 can apply an electronic character recognition conversionto the image data for generating the text data from the image data.

The resource received by the electronic learning system 30 may includemetadata in respect of the content data. The metadata can define variousaspects of the resource, such as, but not limited to, a length of theresource, an identity of the author, the time required for preparing theresource and/or a format of the resource. For example, the metadata caninclude resource structure fields that define a content structure of thecontent data. The content data can be organized in one or more datahierarchy levels and the content structure can indicate the number ofdata hierarchy levels in which the content data is organized.

For example, a Wikipedia article is generally organized by headings. Theheadings can be organized into different data hierarchy levels, such asone or more subheadings. Similarly, an electronic book can be organizedinto various data hierarchy levels, such as a section, a chapter and/orheadings.

At 620, the resource mapping component 112 sections the content datainto one or more content portions based on an analysis of at least oneof the content data and the one or more resource property fields.

The content data provided within each resource may be organized into oneor more content portions. The content portions may correspond to datahierarchy levels, or may correspond to a formatting of the resource(e.g., paragraph structure). The resource mapping component 112 cansection the content data into the content portions based on theorganization of the content data. By sectioning the content data, thescope of each resulting content portion can be narrowed so that theresource mapping component 112 can assign that content portion to alearning objective with an acceptable degree of correlation.

In some embodiments, the resource mapping component 112 may identify thecontent portions within the content data based on the text dataassociated with the content data. For example, the text data can includecertain terminologies associated with hierarchy levels, such as“section”, “chapter”, “part”, etc., and/or numbers representing thecontent portions.

In embodiments when the resource mapping component 112 determines thatthe metadata includes the resource structure fields, the resourcemapping component 112 can section the content data according to at leastone data hierarchy level defined by the resource structure fields. For aWikipedia article, the resource mapping component 112 can determine fromthe resource structure fields associated with the Wikipedia article thatthe Wikipedia article is organized into three headings with each headingbeing further divided into two subheadings. The resource mappingcomponent 112 may section the Wikipedia article into content portionsthat each correspond to the various subheadings.

In some embodiments, a content portion can include two or more datahierarchy levels. The resource mapping component 112, therefore, may notsection the content data according to the lowest data hierarchy level.For example, for a content data with three different data hierarchylevels, the resource mapping component 112 may determine that theoverall length of the content data is relatively short and therefore,sectioning the content data according to the top data hierarchy levelmay be sufficient so that the content of the resulting content portionsmay include two data hierarchy levels.

At 630, the resource mapping component 112 assigns at least one contentportion to at least one learning objective.

As noted, it may be possible that only some of the received resource isrelevant to the learning objectives stored in the learning objectivesdatabase 140, or a content portion is relevant to multiple learningobjectives. The resource mapping component 112 can analyze each of thecontent portions to identify a relevant learning objective.

To determine whether a content portion can be assigned to any learningobjectives, the resource mapping component 112 can apply a semanticanalysis to that content portion as well as to each learning objectivein the learning objectives database 140. In some embodiments, a semanticanalysis component can be operated by the system processor 110 toconduct the semantic analysis on the content portion and/or the learningobjective.

The semantic analysis can involve a comparison of the language used inthe contents to be compared. For example, the semantic analysis caninvolve a comparison of the language in the content portion with thelanguage of each learning objective to determine whether the contentportion is relevant to the learning objective. From the semanticanalysis of the content portion and the learning objective, the resourcemapping component 112 can assign a relevance score to indicate anestimated degree of correlation of the content portion with the learningobjective. The relevance score may be a numerical value.

Each learning objective can be associated with a relevance thresholdvalue. The relevance threshold value can be a minimum relevance scorerequired for a content portion to be associated with that learningobjective. The relevance threshold value may be a numerical value. Therelevance threshold value may be a default value set by the electroniclearning system 30 or predefined by a provider and/or a user of theelectronic learning system 30 in some embodiments. The relevancethreshold value can be stored in association with the learning objectivein the learning objectives database 140.

The resource mapping component 112 can then compare the relevance scorewith the relevance threshold value of the learning objective at issue.When the resource mapping component 112 determines that the relevancescore at least satisfies the relevance threshold value, the resourcemapping component 112 can assign the content portion to that learningobjective since the estimated degree of correlation of the contentportion exceeds the minimum required degree of correlation. However,when the resource mapping component 112 determines that the relevancescore does not satisfy the relevance threshold value, the resourcemapping component 112 can determine that the content portion is notrelevant to the learning objective and will not assign that contentportion to the learning objective.

To assign the content portion with the corresponding learning objective,the resource mapping component 112 may store a reference to the contentportion in association with that learning objective in the learningobjectives database 140. For example, when a resource is an electronicbook and the resource mapping component 112 determines that a certaincontent portion within the electronic book is to be assigned to alearning objective, the resource mapping component 112 can associatethat learning objective with an identifier corresponding to theelectronic book and the relevant page range of the content portion. Whenan action within a learning path involves accessing that content portionwithin the electronic book, the electronic learning system 30 canprovide access directly to the identified relevant page range instead ofto the electronic book as a whole.

Although FIG. 7 involves assigning one or more content portions torelevant learning objectives based on, at least, the analysis applied tothe content portion and the learning objectives, there are embodimentsdescribed herein which can involve resources that are not sectioned andare instead, assigned to learning objectives in their entirety. It willbe understood that the methods described herein, unless specificallystated otherwise, can involve resources that are sectioned and resourcesthat are not sectioned.

Each learning path includes a series of actions in respect of one ormore resources. As described with reference to FIG. 6, the learning pathcomponent 114 can generate learning paths based on, at least, thelearning objectives 320, 330. The learning path component 114 canidentify resources, or, in some embodiments, content portions, that arerelevant to the learning objectives 320, 330. The learning pathcomponent 114 can identify the resources based on data stored in atleast one of the learning objectives database 140 and the resourcesdatabase 142.

In some embodiments, the learning path component 114 can also providethe learning path based on other data, such as data associated withusage of the electronic learning system 30 and/or user data (e.g., userpreferences, etc.).

For example, for a user, the learning path component 114 can provide thelearning path 412, 512 based on the user preference data stored in theuser database 144 for that user. The user preference data can include apreference in the content type to be included in the learning path, suchas text, video, image, etc. The learning path component 114 can providethe learning path 412 by assigning a greater priority to the preferredcontent types. As a result, the learning path component 114 canfacilitate the learning of the user since not all users learn in thesame manner—some users may be a visual learner, some users may be moretextual learners, and other users may be more practical learners whorequire application of the materials.

An example method of providing the learning path will now be describedwith reference to FIGS. 8A and 8B.

Referring now to FIG. 8A, a flowchart diagram illustrating an examplemethod 700 for providing a learning path 512 for the electronic learningsystem 30 is shown. To illustrate the method 700, reference will be madesimultaneously to FIGS. 6 and 9.

At 710, the learning path component 114 retrieves a set of learningobjectives 320, 330 assigned to the learning path 512 for the user.

The set of learning objectives 320, 330 may be previously selected bythe user, or assigned to a group of individuals (including the user) bya third party (e.g., an employer, a teacher, etc.). As described withreference to FIG. 4, the set of learning objectives 320, 330 can bereceived via the user interface 310 and stored in the learningobjectives database 140 in association with the user. The learning pathcomponent 114 can then retrieve the set of learning objectives 320, 330from the learning objectives database 140 for generating the learningpath 512 for the user.

At 720, for each learning objective of the set of learning objectives320, 330, the learning path component 114 selects one or more resourcesassigned the relevance score that at least satisfies the relevancethreshold for that learning objective.

When the learning path component 114 selects the resources for thelearning objectives 320, 330, the learning path component 114 cannotrely on any usage data from the learning path database 146 since theinitial learning path 512 has not previously been provided to any otheruser and therefore, no interaction with the initial learning path 512has occurred. As a result, the learning path component 114 can selectthe resources based on estimated correlations between the resources andthe learning objectives 320, 330.

Similar to 630 of FIG. 7 when the resource mapping component 112 appliesthe semantic analysis to the content portion and the learning objectivesto determine whether to assign the content portion to that learningobjective, the learning path component 114 can select the resourcesbased on whether the relevance score assigned to the resource at leastsatisfies the relevance threshold assigned to the learning objectives320, 330 retrieved. The learning path component 114 can determine therelevance score by conducting a semantic analysis of the learningobjective and the content data of the resource. As described, therelevance score generally indicates an estimated degree of correlationof the resource with the learning objective.

When the learning path component 114 determines that the relevance scoreof a resource at least satisfies the relevance threshold of a learningobjective 320, 330, the learning path component 114 can select thatresource for the learning path 512.

At 730, the learning path component 114 generates an initial learningpath 512 using the selected one or more resources.

As shown in FIG. 6, the sets of actions 530, 540 and 560 include actionsin respect of resources that are associated with a relevance score thatat least satisfies the relevance threshold of the learning objectives320, 330 retrieved. The selected resources are resources that thelearning path component 114 estimates to have a high degree ofcorrelation with the retrieved set of learning objectives 320, 330.

At 740, the system processor 110 identifies one or more evaluationresources from the selected resources.

An evaluation resource is a resource that involves some interactionbetween the user and the electronic learning system 30 in order toevaluate a proficiency of the user with at least a subset of learningobjectives of the set of learning objectives 320, 330. Exampleevaluation resources can include, but are not limited to, assignments,projects, and assessments (e.g., exams, etc.). For example, in theinitial learning path 512 shown in FIG. 6, the example evaluationresources include the resources associated with the actions 534 and 536(“Assignment I” and “Assessment I”, respectively), the action 550(“Assessment II”) and the action 568 (“Assessment III”) respectively.

Each evaluation resource can include one or more questions related to asubset of the set of learning objectives 320, 330 in order to evaluatethe proficiency of the user with that subset of learning objectives 320,330. For example, the evaluation resource associated with the action 550(“Assessment II”) corresponds to the second group 520 b, which isrelated to the subject specific learning objectives 330 a and 330 b, andthe overall learning objective 320 a, and the evaluation resourceassociated with the action 568 (“Assessment III”) corresponds to thethird group 520 c, which is related to the third subject specificlearning objective 330 c, and the overall learning objective 320 b.

At 750, the system processor 110 monitors a feedback usage indicator foreach evaluation resource.

The feedback usage indicator can generally represent an amount of userinteraction with that evaluation resource. The feedback usage indicatorcan be represented as a numerical value, in some embodiments. Thefeedback usage indicator can be stored in the learning path database146, in some embodiments.

With each use of the evaluation resource in the learning path 512, thecorresponding feedback usage indicator increases in value and theelectronic learning system 30 can also collect usage data related tothose interactions by the users with the evaluation resources.

At 760, the learning path component 114 updates the initial learningpath 512 to generate a learning path based on, at least, the feedbackusage indicator determined for each evaluation resource.

From the interactions with the learning path 512 by the one or moreusers and, in particular, with the evaluation resources, the electroniclearning system 30 can obtain usage data indicative of a system learnvalue of the resources selected at 720. The system learn value for theresources in the learning path can represent a likelihood that aparticular resource will assist the users in achieving the correspondinglearning objective. That is, the system learn value can represent anextent of user data the electronic learning system has obtained inrespect of the resource. The system learn value may be a numerical valuecorresponding to a probability of that particular resource to assist theusers in achieving the learning objective.

The electronic learning system 30 can assign a system learn value toeach resource selected at 720 in response to each time a user completesa corresponding evaluation resource. When an evaluation resourcereceives more than one system learn value (e.g., the evaluation resourceis used more than once), the electronic learning system 30 may assignthe system learn value as an average or a sum of the multiple systemlearn values received after each use of that evaluation resource, orselect the highest or lowest of each of the system learn values. Theelectronic learning system 30 may track each of the system learn valuesreceived in the learning path database 140.

Generally, the learning path component 114 can update the initiallearning path 512 based on the system learn value assigned to eachresource in the initial learning path 512. Reference will now be made toFIG. 6 and FIG. 9. FIG. 9 is a screenshot 900 of an example userinterface 910 showing an updated version of the initial learning path512 shown in FIG. 6, or the learning path 912.

The learning path component 114 can, in some embodiments, generate thelearning path 912 by rearranging an order in which the actionsassociated with the resources are presented in the initial learning path512. In one example, the actions associated with the resources can berearranged according to the respective system learn values assigned toeach resource in a decreasing numerical sequence so that those actionsassociated with resources assigned a higher learn order appears first inthe learning path 912 with the actions associated with resourcesassigned a lower learn order following thereafter.

For example, as shown in FIG. 9, several actions within the first seriesof actions 530 and the third series of actions 560 in the learning path512 of FIG. 6 are now rearranged in the learning path 912.

Briefly, the learning path 912 now includes a first series of actions930 associated with the first group 520 a (“Marketing Basics”), a secondseries of actions 940 associated with the second group 520 b(“Products”), a third series of actions 950 associated with the thirdgroup 520 c (“Product & Pricing”), and a fourth series of actions 960associated with the fourth group 920 (“Final”). Each of the first,second, third and fourth sets of actions 930, 940, 950, and 960 includesactions in respect of various different resources.

In comparison with the first series of actions 530 in FIG. 6, the firstseries of actions 930 in the learning path 912 includes only the action532. The learning path component 114, based on the system learn valuesassigned to the resources, namely Assignment I and Assessment I, hasrearranged the corresponding actions 534 and 536 as actions 962 and 964,respectively, within the fourth series of actions 960. For theembodiment shown in FIG. 9, the learning path component 114 candetermine that the system learn values assigned to the resourcesassociated with the actions 534 and 536 are lower than the system learnvalues assigned to the other resources within the initial learning path512. The learning path component 114, therefore, rearranged the actions534 and 536 to appear at the end of the learning path 912.

The third series of actions 950 includes the actions 564 and 566 in areversed order, namely as actions 952 and 954, respectively. Similar tothe rearrangement of the actions 534 and 536, the learning pathcomponent 114 can rearrange actions 564 and 566 in response todetermining that the system learn value assigned to the resourceassociated with the action 564 is less than the system learn valueassigned to the resource associated with the action 566.

In some embodiments, the learning path component 114 can generate thelearning path by removing at least one resource from the initiallearning path 512. For example, the learning path component 114 mayinclude in the learning path resources with a system learn value that atleast satisfies a learn value threshold. The learn value threshold canbe a system learn value required for the resource to be retained in thelearning path. The learning path component 114 can then update theinitial learning path 512 by determining whether the system learn valueassigned to each resource is less than the learn value threshold and ifthe system learn value is determined to be less than the learn valuethreshold, the learning path component can remove the action associatedwith that resource from the initial learning path 512.

In some embodiments, if the system learn value is determined to be lessthan the learn value threshold, the learning path component can add adifferent action to the initial learning path 512. The learning pathcomponent 114 may, in some embodiments, update the initial learning path512 by removing the action associated with that resource and adding thedifferent action to the initial learning path 512.

For example, in comparing FIG. 6 with FIG. 9, the actions 546 and 548have been removed from the second series of actions 540 in FIG. 6. Thesecond series of actions 940 in the learning path 912 now only includesthe actions 542, 544 and 550. The learning path component 114 may haveremoved the actions 546 and 548 in response to determining that thesystem learn values assigned to the resources corresponding to theactions 546 and 548 do not satisfy the system learn value threshold, andtherefore, those resources do not sufficiently contribute to thelearning of the users to justify being included in the learning path912.

To benefit from as much usage data as is reasonable so that the learningpath component 114 can update the initial learning path 512 with areliable system learn value, the learning path component 114 may, insome embodiments, update the initial learning path 512 when the feedbackusage indicator reaches certain predefined thresholds. As will bedescribed with reference to FIG. 8B, which is a flowchart diagramillustrating an example method 800 for updating the initial learningpath 512, the learning path component 114 may update the initiallearning path 512 differently depending on the predefined threshold thatis met.

At 810, the learning path component 114 determines, for each evaluationresource, whether the feedback usage indicator at least satisfies aninitial usage threshold.

The initial usage threshold can be a minimum amount of usage of anevaluation resource in order for the learning path component 114 toupdate the initial learning path 512 based at least partially on thesystem learn values assigned to the resources selected at 720.

If the learning path component 114 determines that the feedback usageindicator does not satisfy the initial usage threshold, the learningpath component 114 proceeds to 850. At 850, the learning path component114 provides the initial learning path 512 as the learning path 912since the amount of use of the evaluation resource is not sufficient forupdating the initial learning path 512 and therefore, the learning pathcomponent 114 retains the initial learning path 512.

If the learning path component 114 determines that the feedback usageindicator at least satisfies the initial usage threshold at 810, thelearning path component 114 can proceed to 820.

It will be understood that, in some embodiments, when the learning pathcomponent 114 determines, at 810, that the feedback usage indicator atleast satisfies the initial usage threshold, the learning path component114 can proceed directly to 840 without determining whether the feedbackusage indicator at least satisfies a subsequent usage threshold.

At 820, the learning path component 114 determines whether the feedbackusage indicator at least satisfies the subsequent usage threshold.

The subsequent usage threshold is generally greater in value than theinitial usage threshold. The subsequent usage threshold can be a minimumamount of usage of an evaluation resource in order for the learning pathcomponent 114 to update the initial learning path 512 based entirely onthe system learn values assigned to the resources selected at 720. Whenthe learning path component 114 determines that the feedback usageindicator at least satisfies the subsequent usage threshold, the usagedata associated with the initial learning path 512 can be consideredsufficient so that the resulting system learn values are reliable.

If, at 820, the learning path component 114 determines that the feedbackusage indicator at least satisfies the subsequent usage threshold, thelearning path component 114 can proceed to 830. At 830, the learningpath component 114 generates the learning path 912 based on the systemlearn values assigned to the resources selected at 720.

If the learning path component 114 determines, at 820, that the feedbackusage indicator does not satisfy the subsequent usage threshold, thelearning path component 114 can proceed to 840. At 840, the learningpath component 114 can generate the learning path 912 based on acombined score generated using the relevance score and the system learnvalue assigned to each resource selected at 720.

In some embodiments, the learning path component 114 can generate thecombined score by applying a first weight to the relevance score and asecond weight to the system learn value. The combined score may be a sumof the weighted relevance score and the weighted system learn value. Thefirst weight and the second weight may, in some embodiments, each benumerical values that, together, sum to one.

As described with reference to FIGS. 8A to 11, the electronic learningsystem 30 can generate the initial learning path 512 based on estimatedcorrelations between the received learning objectives 320, 330. Theelectronic learning system 30 can then, in accordance to the variousdescribed methods, proceed to update the initial learning path 512 inresponse to the collected usage data to generate the learning path 912.

The electronic learning system 30 may, in some embodiments, modify thelearning path for individual users in response to certain feedbackinputs received prior to the user progressing through the learning path512 or even while the users progress through the learning path 512. Bymodifying the learning path 512 according to the needs and/orpreferences of the individual users, the electronic learning system 30can adapt the learning path 512 to the users and, consequently, furtherenhance the learning experience of each user. Example methods formodifying the learning path 512 will now be described with reference toat least FIGS. 10 to 14.

Referring now to FIG. 10, a flowchart diagram illustrating an examplemethod 1000 for modifying the learning path 512 for the electroniclearning system 30 is shown. To illustrate the method 1000, referencewill be made simultaneously to FIGS. 4, 6, and 11 to 14.

At 1010, the learning path component 114 retrieves a set of learningobjectives 320, 330 assigned to the user.

As described with respect to 710 of FIG. 8A, the learning path component114 can retrieve the set of learning objectives 320, 330 from thelearning objectives database 140 for the user. The set of learningobjectives 320, 330 may be previously selected by the user, or assignedto a group of individuals (including the user) by a third party (e.g.,an employer, a teacher, etc.).

At 1020, the learning path component 114 retrieves the path dataassociated with the learning path 512 defined for the user.

The path data, as described, can be stored in the learning path database146. In some embodiments, the path data can be stored in one or moreother storage components. Generally, the path data includes dataassociated with the learning paths defined by the learning pathcomponent 114. The path data can include, but is not limited to,identifiers corresponding to the resources selected to be included intothe learning path 512, user identifiers corresponding to the users withaccess to the learning path 512, and/or usage data associated with theinteraction with the learning path 512 by the respective users.

At 1030, the electronic learning system 30 receives one or more userresponse inputs from the user in respect of at least one learningobjective of the set of learning objectives 320, 330.

User response inputs may be received via the interface component 116.The electronic learning system 30 can receive the user response inputsin respect of the set of learning objectives 320, 330 in various forms.

For example, the user response inputs may include interaction by theuser with the various evaluation resources included in the learning path512. The evaluation resources, as described with reference to FIG. 6,typically include one or more questions related to a subset of learningobjectives from the set of learning objectives 320, 330 in order toevaluate the proficiency of the user with that subset of learningobjectives 320, 330.

In some embodiments, the electronic learning system 30 may provide anevaluation tool specifically for receiving the user response inputs inrespect of the set of learning objectives 320, 330. The evaluation toolcan be a pretest for determining whether the user is proficient with anyone or more of the learning objectives 320, 330.

Referring now to FIG. 11, a screenshot 1100 of an example user interface1110 providing a portion of an example evaluation tool 1112 forreceiving inputs from the user is shown. As shown in FIG. 11, theevaluation tool 1112 can include a list 1120 of one or more questionsthat represent at least one learning objective 320, 330 and eachquestion in the list 1120 has at least one corresponding response field1130 for receiving the user response input in respect of that questionin the list 1120. For example, the list 1120 includes questions 1122,1124, 1126 and 1128. Each question 1122, 1124 and 1128 has acorresponding response field 1132, 1134 and 1138, respectively, forreceiving the user response input to those questions. Question 1126requires a multi-part response and therefore, has a response field 1136with three separate field components.

Also, the questions 1122, 1124, 1126 and 1128 in the example shown inFIG. 11 generally correspond to the overall learning objective 320 asince the questions 1122, 1124, 1126 and 1128 are related to the topicof product development. The questions 1122 and 1124 are related to theclassification of products, which correspond to the subject specificlearning objective 330 a, and the questions 1126 and 1128 are related tothe product life cycle, which correspond to the subject specificlearning objective 330 b.

It will be understood that FIG. 11 only illustrates a portion of theevaluation tool 1112 and the list 1120 of questions can include one ormore questions that are not shown.

At 1040, the electronic learning system 30 evaluates the received one ormore user response inputs to determine a competence level of the user inrespect of the at least one learning objective 320, 330.

The competence level can indicate a proficiency of the user with the atleast one learning objective. The user response inputs can represent afamiliarity of the user with the various learning objectives 320, 330.

In some embodiments, the evaluation component 118 can generate a scorefor each of the user response inputs received at 1030. The score may bea numerical value indicating whether the user response input is thecorrect answer to the corresponding question. The score may be generatedby applying a mathematical operation to the results of the user responseinputs. For example, the score may be a sum, average or maximum value ofthe results of the user response inputs.

The electronic learning system 30 may then determine whether the scoregenerated by the evaluation component 118 at least satisfies a masterythreshold for that learning objective 320, 330. The mastery thresholdcan be a minimum score generated by the evaluation component 118required for the user to be at the mastery level in respect of aparticular learning objective 320, 330. The mastery threshold may, insome embodiments, be defined by a provider of the electronic learningsystem 30 and/or a user of the electronic learning system 30 (e.g., ateacher, an employer, etc.).

If the electronic learning system 30 determines that the score generatedby the evaluation component 118 at least satisfies the masterythreshold, the competence level of the user is the mastery level.However, if the electronic learning system 30 determines that the scoregenerated by the evaluation component 118 does not satisfy the masterythreshold, the competence level of the user is the satisfactory level.The satisfactory level can indicate that the user is not completelyproficient with that learning objective but is not at an unreasonableproficiency level.

In some embodiments, when the electronic learning system 30 determinesthat the score generated by the evaluation component 118 does notsatisfy the mastery threshold, the electronic learning system 30 canfurther determine whether the score generated by the evaluationcomponent 118 satisfies a satisfactory threshold. The satisfactorythreshold can generally be less than the mastery threshold andcorresponds to a reasonable proficiency of the user in respect of thatlearning objective. When the electronic learning system 30 determinesthat the score generated by the evaluation component 118 satisfies thesatisfactory threshold, the competence level of the user is thesatisfactory level. However, if the electronic learning system 30determines that the score generated by the evaluation component 118 doesnot satisfy the satisfactory threshold, the competence level of the useris a deficient level.

In some embodiments, the mastery threshold can require all questions inrespect of a learning objective to be correctly answered by the user.

At 1050, the learning path component 114 modifies the learning path 512for the user based on the competence level determined for the user inrespect of the at least one learning objective 320, 330.

Based on the determinations of the competence level made at 1040, thelearning path component 114 can modify the learning path 512accordingly.

Reference is now made to FIG. 12, which is a screenshot 1200 of anexample user interface 1210 showing a version of the learning objectives320, 330 based on the user inputs received via the user interface 1110in FIG. 11.

When the learning path component 114 determines that the competencelevel of the user in respect of at least one learning objective 320, 330is the mastery level, the learning path component 114 can assign thatlearning objective with a mastery status. The mastery status indicatesthat the user is proficient with that learning objective 320, 330 andtherefore, minimal action, and possibly even no action, associated withthat learning objective 320, 330 is required in the learning path 512.When the learning objective 320, 330 is assigned the mastery status, thelearning path component 114 can assign a mastery status to that learningobjective 320, 330 and modify the learning path 512 based on thestatuses of each of the learning objectives 320, 330 accordingly.

For example, as shown in FIG. 12, the learning objectives 1220 a and1230 b have been assigned the mastery status 1240 and 1242,respectively.

From the user response inputs received in respect of the list 1120 ofquestions shown in FIG. 11, the evaluation component 118 can determinethat the user response inputs received in the response fields 1132,1134, 1136 and 1138 corresponding to the questions 1122, 1124, 1126, and1128 satisfy the mastery threshold. As a result, since the questions1122, 1124, 1126, and 1128 correspond to the overall learning objective1220 a, the learning path component 114 can assign the mastery status1240 to the overall learning objective 1220 a. Similarly, sincequestions 1126 and 1128 correspond to the subject specific learningobjective 1230 b (assigned the mastery status 1242), the evaluationcomponent 118 has also determined, in this example, that the userresponse inputs received in the respective response fields 1136 and 1138satisfy the mastery threshold.

As a result, at least some of the learning objectives in the set oflearning objectives 1220, 1230 are associated with a mastery status, asshown in FIG. 12. The learning path component 114 can then modify thelearning path 512 based on the status of each of the learning objectivesin the set of learning objectives 1220, 1230.

In some embodiments, one or more learning objectives 320, 330 can beassigned a mandatory status. The mandatory status can indicate that theactions in the learning path 512 associated with that learning objectiveare required for the user, and cannot be removed from the learning path512 despite the electronic learning system 30 determining that the userhas reached the mastery level in respect of that learning objective.That is, the mandatory status can override the effects of the masterystatus 1240, 1242. The learning path component 114, therefore, canretain the actions associated with the learning objective 320, 330assigned the mandatory status within the learning path 512.

For example, in FIG. 12, the subject specific learning objective 1230 a,which corresponds to the subject specific learning objective 330 a, isassigned the mandatory status 1250. As described with respect to FIG.11, the questions 1122 and 1124 are related to the subject specificlearning objective 330 a. Since the evaluation component 118 determined,in the example of FIG. 12, that the user response inputs received in thecorresponding response fields 1132 and 1134 satisfy the masterythreshold, the learning path component 114 can assign a mastery statusto the subject specific learning objective 330 a. However, due to themandatory status 1250 also assigned to the subject specific learningobjective 330 a, the learning path component 114 cannot remove theactions associated with the subject specific learning objective 330 a.

Continuing still with FIG. 12, in some embodiments, the learning pathcomponent 114 can determine that the competence level of the user inrespect of at least one learning objective 320, 330 is the deficientlevel. The user is assigned the deficient level for a learning objective320, 330 when the electronic learning system 30 determines that theresponse inputs received in respect of questions associated with thelearning objective 320, 330 fails to satisfy the satisfactory threshold.The learning path component 114 can, as a result, assign that learningobjective 320, 330 with an alert status to indicate that the user mayrequire additional training in respect of that learning objective 320,330.

In FIG. 12, the overall learning objective 1220 b is assigned the alertstatus 1254. When the learning path component 114 assigns the overalllearning objective 1220 b with the alert status 1254, the learning pathcomponent 114 can identify additional actions related to the overalllearning objective 1220 b but are not already in the learning path 512.The additional actions can be different from the actions in the learningpath 512 that are already being conducted. The learning path component114 can then assign those additional actions with a recommendedindicator to indicate that those additional actions are suggested forthe user to improve his or her competence level but may not be requiredfor the user to arrive at the mastery level. The electronic learningsystem 30, therefore, facilitates the user to improve on certainlearning objectives 320, 330 but at the user's discretion.

Based on the above modifications and characterization of the learningobjectives 320, 330, at least some of the learning objectives in the setof learning objectives are now associated with different statuses, asshown in FIG. 12. The overall learning objectives 1220 include a firstoverall learning objective 1220 a (corresponding to the first overalllearning objectives 320 a of FIG. 4) assigned the mastery status 1240and a second overall learning objective 1220 b (corresponding to thesecond overall learning objectives 320 b of FIG. 4). The subjectspecific learning objectives 1230 includes a first subject specificlearning objective 1230 a (corresponds to the first subject specificlearning objective 330 a), a second subject specific learning objective1230 b (corresponds to the second subject specific learning objective330 b from FIG. 4) assigned the mastery status 1242, a third subjectspecific learning objective 1230 c (corresponds to the third subjectspecific learning objective 330 c), and a fourth subject specificlearning objective 1230 d (corresponds to the fourth subject specificlearning objective 330 d). For ease of exposition, indicators are notshown in FIG. 12 for illustrating the satisfactory status associatedwith the learning objectives 1220 b, 1230 c and 1230 d.

The learning path component 114 can then modify the learning path 512for the user based on the status assigned to each learning objective ofthe set of learning objectives 1220, 1230. An example learning path 1312will now be described with reference to FIG. 13, which is a screenshot1300 of an example user interface 1310 showing the learning path 1312generated based on the learning objectives 1220, 1230 in FIG. 12.

The learning path 1312 includes a first series of actions 1330associated with the second group 520 b of the learning path 512, asecond series of actions 1340 associated with the third group 520 c ofthe learning path 512 and a third series of actions 1350 associated witha new group 1320 (“Promotion of Products and Services”).

Unlike the learning path 512 of FIG. 6, the learning path 1312 does notinclude the first series of actions 530 associated with the first group520 a. When generating the learning path 512 based on the learningobjectives 1220, 1230, the learning path component 114 can determinethat the learning path 1312 does not require the first series of actions530 associated with the first group 520 a since the overall learningobjective 320 a is assigned the mastery status 1240 and the first seriesof actions 530 is generally introductory content directed at gainingsome feedback data from the user via the actions 534 and 536.

Also, based on the alert status 1254 assigned to the overall learningobjective 1220 b, the learning path component 114 generated the learningpath 1312 to include the third series of actions 1350 associated withthe new group 1320. As shown in FIG. 13, each of the actions 1352, 1354,and 1356 in the third series of actions 1350 is assigned the recommendedindicator 1360 to indicate that those actions 1352, 1354, and 1356 areonly recommendations.

Unlike the learning path 512, the first series of actions 1330associated with the second group 520 b includes only the actions 542 and550, and does not include the actions 544, 546 and 548. The learningpath component 114 has removed the actions 544, 546 and 548 from thelearning path 1312 since the corresponding learning objective 330 b isassigned the mastery status 1242.

Reference is now made to FIG. 14, which is a screenshot 1400 of anexample user interface 1410 showing another example learning path 1412generated based on the learning objectives 1220, 1230 in FIG. 12.

As can be seen from FIGS. 13 and 14, the actions in the learning path1412 are generally similar to the learning path 1312 except the learningpath 1412 includes the first series of actions 530 associated with thefirst group 520 a from FIG. 6. Unlike the learning path 1312, thelearning path component 114 generated the learning path 1412 in responseto receiving user response inputs while the user was progressing throughthe learning path 512 of FIG. 6 so that a first portion of the learningpath 512 has already been completed by the user while a remainingportion of the learning path 512 is incomplete.

In the example of FIG. 14, the user response inputs can be received aspart of the action 536 when the user accesses the resource, AssessmentI. The content of Assessment I may correspond to the content of theevaluation tool 1112 of FIG. 11. The first series of actions 530 iscomplete but the remaining sets of actions 540 and 560 of FIG. 6 arenot.

Based on the response user inputs received via the action 536 in respectof the “Assessment I”, the electronic learning system 30 can evaluatethose received response user inputs according to 1040 of FIG. 10. Thelearning path component 114 can then modify the remaining portion of thelearning path 512, such as the remaining series of actions 540 and 560,based on the competence levels determined at 1040. To modify theremaining portion of the learning path 512, the learning path component114 can determine whether any action of the series of actions 540 and560 corresponds to the learning objectives 320 a and 330 b, which havebeen assigned the mastery status 1240 and 1242, respectively.

For ease of exposition, the example learning path 1412 shown in FIG. 14is modified by the learning path component 1114 on the assumption thatthe content of the “Assessment I” corresponds to the list 1120 ofquestions so that the response user inputs received via the action 536will be the same as the response user inputs received in respect of thelist 1120 in FIG. 11, and therefore, the set of learning objectivescorrespond to the learning objectives shown in FIG. 12. The remainingseries of actions 540 and 560 of the learning path 512 is modified inthe same manner as the learning path 1312 shown in FIG. 13. The learningpath 1412, therefore, includes the series of actions 530, 1330, 1340 and1350.

Referring now to FIG. 15, which is a flowchart diagram of a method 1500for providing an evaluation resource for a user 12, 14 of the electroniclearning system 30. Reference will be made to, at least, FIGS. 15 to 17for illustrating the method 1500. The method 1500 can be used, in someembodiments, to dynamically generate evaluation resources during theuser's interaction with a learning path 1612. The example learning path1612 corresponds to the learning path 512 shown in FIG. 6. Unlike thelearning path 512, a dynamically generated evaluation resource can begenerated by the electronic learning system 30 when the user 12, 14activates the action 1650 in the learning path 1612.

With dynamic evaluation resources, the electronic learning system 30 maycustomize the evaluation resources. For example, the electronic learningsystem 30 may customize the evaluation resource based on the user'sprior interaction with the learning path 1612, such as recent actionswithin the learning path 1612, strengths and/or weaknesses demonstratedin the prior interactions, and/or characteristics of prior actions inrespect of certain resources (e.g., a length of time spent on an actionin respect of a particular resource, etc.). Other factors may similarlyaffect how the content of the evaluation resource may be customized. Byproviding evaluation resources based on, at least, the user'sinteraction with the learning path 1612, the electronic learning system30 can obtain usage data that is representative of the user's currentknowledge in respect of the various learning objectives 320, 330associated with the learning path 1612. With the data resulting from theuser's interaction with the evaluation resource, the electronic learningsystem 30 may then determine whether the remainder of the learning path1612 requires adjustments in order to accommodate the learning of thatuser 12, 14.

At 1510, the system processor 110 identifies, from the one or more userprofiles, a user profile associated with the user 12.

As described, the user database 144 can store user profiles for each ofthe users 12, 14 of the electronic learning system 30. Each user profilecan include user interaction data that represents the user's interactionwith the relevant learning paths provided by the electronic learningsystem 30.

For example, when the user 12 interacts with the learning path 1612, theuser database 144 can store user interaction data corresponding to thevarious actions conducted by the user 12 in respect of the learning path1612, such as when the user 12 conducts the action 542 (e.g., readingthe article regarding product types) in the second series of actions1640 in the learning path 1612. The user interaction data correspondingto the action 542 can include a time and date of when the action 542 wasinitiated, and/or a length of time the user 12 spent on the action 542.Other data in respect of the action 542 may similarly be stored as theuser interaction data in the user database 144.

Similarly, when the user 12 initiates the action 1650 (e.g., conducting“Assessment II”), the user database 144 can store user interaction dataindicating that the user 12 has submitted an evaluation request. Basedon the evaluation request, the system processor 110 can determine thatthe evaluation resource is to be generated and to be provided to theuser 12.

At 1520, the system processor 110 detects, from the user interactiondata in the identified user profile, an evaluation request from the userin respect of the learning path 1612.

The evaluation request may be associated with at least one resource inthe learning path 1612. For example, the evaluation request can beassociated with the resources 1680 and 1682, namely the “Product TypeArticle” and the “Life Cycle Article”. In response to the evaluationrequest, the system processor 110 may then generate the evaluationresource based on the content of each of resources 1680 and 1682.

In some embodiments, the evaluation request may be generated in responseto actions in the learning path 1612 that are not directly related toevaluation resources. For example, when the user 12 selects the action542 in respect of the resource 1680, the action 542 can include asubmission of an evaluation request to the electronic learning system30.

At 1530, the system processor 110 determines whether an evaluationrequest has been detected.

When the system processor 110 has not detected the evaluation request,the system processor 110 can proceed to 1550 to continue to monitor theuser interaction data for any evaluation request. However, when thesystem processor 110 detects the evaluation request, the systemprocessor 110 can proceed to 1540. In the example shown in FIG. 16, asbriefly described, the system processor 110 can receive the evaluationrequest when the action 1650 is selected.

At 1540, the system processor 110 generates the evaluation resourcebased on, at least, resource data corresponding to the at least oneresource associated with the evaluation request.

FIG. 17 is a screenshot 1700 of an example user interface 1710 providingan evaluation resource 1712. The evaluation resource 1712 shown in FIG.17 is a dynamic pop quiz. Alternatively, the evaluation resource 1712can be provided by the electronic learning system 30 as practice studyquestions for the user 12. The evaluation resource 1712 can include oneor more evaluation items 1720 for evaluating the user 12.

As shown in FIG. 17, the evaluation items 1720 can include a set ofquestions with a corresponding set of response fields 1730 for receivinguser response inputs from the user 12. The example evaluation resource1712 shown in FIG. 17 includes four evaluation items 1720 related to theresource 1680, namely evaluation items 1722, 1724, 1726 and 1728. It ispossible for the evaluation resource 1712 to include a different numberof evaluation items 1720. In some embodiments, the number of evaluationitems 1720 in the evaluation resource 1712 can be restricted to adesired total number as predefined by the user 12, provider of thelearning path 1612 (e.g., teacher, etc.) and/or operator of theelectronic learning system 30.

To generate the evaluation resource 1712, the system processor 110 mayselect the evaluation items 1720 from a predefined set of evaluationitems stored in the resources database 142 using at least some of themethods described herein. The resources database 142, for example, canstore multiple different evaluation items for each of the learningobjectives 230, 330. In some embodiments, the resources database 142 caninclude an evaluation item database, or be in electronic communicationwith an evaluation item database, that is dedicated to storing theevaluation items. The evaluation item database can include one or moredatabases, and the one or more databases may be distributed in one ormore different geographical locations.

The evaluation item database can be updated, from time to time, by theprovider of the learning path 1612 (e.g., teacher, etc.) and/or operatorof the electronic learning system 30. Updates to the evaluation itemdatabase can include removal of evaluation items that the provider oroperator considers to no longer be useful (e.g., overly used, no longerrelevant, unsuitable, etc.) and/or addition of evaluation items.

As shown in FIG. 17, evaluation items can include questions that areintended to evaluate the user's 12 knowledge in respect of a learningobjective 230, 330. Other forms of the evaluation items may beavailable.

For example, the system processor 110 may conduct a semantic analysisbetween each of the evaluation items stored in the resources database142 and the resource data associated with the resource 1692 to determinea relevance of each of the stored evaluation items to the resource 1692.As described, in some embodiments, a semantic analysis component can beoperated by the system processor 110 to conduct the semantic analysis.The semantic analysis may include a comparison of the content of eachresource data with a content of each evaluation item stored in theresources database 142.

The resource data can include, at least, a resource content of theresource 1680 (e.g., a content of the “Product Type Article”) and/or thelearning objectives for which the resource 1680 was included in thelearning path 1612 to fulfill. These learning objectives can be referredto as resource learning objectives. As described above with respect toFIG. 4, the resource 1680 is included in the learning path 1612 toaddress, at least, the learning objective 330 a (the learner is requiredto be able to classify products according to type). Therefore, thesemantic analysis can, in some embodiments, include a comparison of theresource content with the content of each evaluation item stored in theresources database 142, and/or a comparison of the content of eachresource learning objective with the content of each evaluation itemstored in the resources database 142.

Based on the results of the semantic analysis, the system processor 110can generate a relevance score for each evaluation item in the resourcesdatabase 142 to indicate an estimated degree of correlation of thatevaluation item to the resource content and/or each learning objective.For example, the system processor 110 can generate a resource relevancescore for each evaluation item based on the semantic analysis of theresource content with the content of each evaluation item. The systemprocessor 110 can also generate a learning objective relevance score foreach evaluation item based on the semantic analysis of the content ofeach resource learning objective with the content of each evaluationitem. Each of the resource relevance score and the learning objectiverelevance score is representative of an estimated degree of relevancebetween a particular evaluation item stored in in the resources database142 and the resource content of the resource 1692 or the content of thelearning objective 330 a, respectively.

As shown in FIG. 17, the evaluation items 1722 and 1726 are directed toclassifying products X and Y, respectively, and the evaluation items1724 and 1728 are directed to the characteristics of the class of eachof products X and Y, respectively.

To select evaluation items 1722 to 1728 for the evaluation resource1712, the system processor 110 can conduct a semantic analysis of therelevant contents. For example, the system processor 110 can conduct asemantic analysis of the resource content of the resource 1680, which isan article regarding product types, with each evaluation item stored inthe evaluation item database. The system processor 110 can assign aresource relevance score to each evaluation item in the evaluation itemdatabase based on the semantic analysis. Similarly, the system processor110 can conduct a semantic analysis of the content of the learningobjective 330 a, which requires that the learner be able to classifyproducts according to type, with each evaluation item stored in theevaluation item database. The system processor 110 can assign a learningobjective relevance score to each evaluation item in the evaluation itemdatabase based on the semantic analysis.

In some embodiments, the system processor 110 may conduct the semanticanalysis in respect of a sub-group of evaluation items in the evaluationitem database. For example, the evaluation item database can include asub-group of evaluation items that is directed to productclassifications. Other sub-groups may be available. An evaluation itemcan be assigned to one or more different sub-groups.

The system processor 110 can select the evaluation items from theevaluation item database for the evaluation resource 1712 based on theresource relevance score and the learning objective relevance score. Forexample, the system processor 110 can select the evaluation items forthe evaluation resource 1712 by selecting the evaluation items with thehighest resource relevance score and highest learning objectiverelevance score.

In some embodiments, the system processor 110 can combine the resourcerelevance score and the learning objective relevance score to generatean overall relevance score. The resource relevance score and thelearning objective relevance score may be combined by applyingrespective weights, such as applying a resource weight to the resourcerelevance score and a learning objective weight to the learningobjective relevance score. The overall relevance score can be a sum ofthe weighted resource relevance score and the weighted learningobjective relevance score. For example, after conducting the semanticanalysis, the system processor 110 can determine an overall relevancescore for each of the evaluation items in the evaluation item database.The system processor 110 can select the evaluation items with thehighest overall relevance score, such as the evaluation items 1722 to1728 in the example shown in FIG. 17, for the evaluation resource 1712.

The sum of the resource weight and the learning objective weight can beone. For example, each of the resource weight and the learning objectiveweight can be 0.5. The value of the resource weight and the learningobjective weight can be varied by the user 12, the operator of theelectronic learning system 30, and/or the provider of the learning path1612.

The system processor 110 can then generate the evaluation resource 1712based on the relevance scores assigned to at least some of theevaluation items stored in the resources database 142.

In some embodiments, the system processor 110 can generate theevaluation resource 1712 by selecting the evaluation items stored in theresources database 142 with a relevance score that exceeds a relevancethreshold value. The evaluation items with a relevance score exceedingthe relevance threshold value can be referred to as the relevantevaluation items. The relevance threshold value can be a minimumrelevance score required for that evaluation item to be included in theevaluation resource 1712. The system processor 110 can include theevaluation items stored in the resources database 142 with a relevancescore that exceeds the relevance threshold value into the evaluationresource 1712 and exclude all evaluation items stored in the resourcesdatabase 142 with a relevance score that is equal or less than therelevance threshold value. Referring again to the example shown in FIG.17, prior to selecting the evaluation items 1722 to 1728, the systemprocessor 110 can first determine whether the respective overallrelevance scores exceed the relevance threshold value and include theevaluation items 1722 to 1728 in response to determining the respectiveoverall relevance scores exceed the relevance threshold value.

The relevance threshold value may vary for different evaluationresources 1712 and/or may be varied by the user 12, the operator of theelectronic learning system 30, and/or the provider of the learning path1612. For example, the relevance threshold value may be adjusted whenthe system processor 110 determines that the number of relevantevaluation items fails to meet the desired number of evaluation items.In those cases, the system processor 110 may automatically decrease therelevance threshold value by a predefined amount or the system processor110 may generate an error message to the operator of the electroniclearning system 30 and/or the provider of the learning path 1612indicating that the relevance threshold value is too high. If the numberof relevant evaluation items is satisfactory, the system processor 110can maintain the predefined relevance threshold value without varyingit.

In some embodiments, the system processor 110 may generate theevaluation resource 1712 with reference to historical usage data storedin the resources database 142 in respect of the selected evaluationitems 1720. For example, each time that a user interacts with anevaluation item in the resource database 142, the system processor 110can update a corresponding system learn value for that evaluation item.The system learn value can be determined based on a frequency in whichother users of the electronic learning system 30 has responded correctlyto that evaluation item. As more and more users interact with theelectronic learning system 30 and the evaluation items in the resourcesdatabase 142, the system learn value can be increasingly representativeof the appropriateness of the evaluation item 1720 in evaluating theknowledge of the user 12 in respect of the relevant learning objections230, 330. The electronic learning system 30 may track each of the systemlearn values in the resources database 142.

The system learn may be a numerical value. For example, the system learnvalue of an evaluation item may be a ratio of a number of correctresponses and a total number of attempts at responding to thatevaluation item.

For example, when the system learn value is high, the system processor110 can determine that the respective evaluation item 1720 may be tooeasy since most users who interacted with that evaluation item 1720 hasresponded correctly. When the system learn value is low, the systemprocessor 110 can determine that the respective evaluation item 1720 maybe too difficult since most users who interacted with that evaluationitem 1720 has responded incorrectly.

When the system processor 110 determines that the system learn valuesare available for the relevant evaluation items, the system processor110 may select a subset of the relevant evaluation items as theevaluation items 1720. For example, the system processor 110 may selectthe evaluation items assigned the system learn value within a medianrange of the overall range of the system learn values as the evaluationitems 1720. The evaluation items assigned the system learn values withinthe median range can be items to which a similar number of usersanswered correctly and incorrectly. Those evaluation items 1720 can bemore suitable for evaluating the user 12

To select the evaluation items with system learn values within themedian range, the system processor 110 can determine a median value forthe system learn values of the relevant evaluation items. The systemprocessor 110 can select the desired number of evaluation items 1720with system learn values closest in value to the median value. Forexample, the system processor 110 can select the evaluation item with afirst system learn value that is closest in value to the median value,and can then select another evaluation item with a second system learnvalue that is next closest in value to the median value but not closerin value to the median value than the first system learn value. Thesystem processor 110 can continue to select the evaluation items 1720from the relevant evaluation items until the desired number ofevaluation items 1720 is met.

The embodiments herein have been described here by way of example only.Various modification and variations may be made to these exampleembodiments. Also, in the various user interfaces illustrated in thefigures, it will be understood that the illustrated user interface textand controls are provided as examples only and are not meant to belimiting. Other suitable user interface elements may be possible.

We claim:
 1. A method for improving resource content mapping for anelectronic learning system, the electronic learning system including aprocessor and at least one memory in electronic communication with theprocessor, the at least one memory storing one or more learningobjectives, the method comprising: receiving, by the electronic learningsystem, a resource for satisfying at least one learning objective of theone or more learning objectives, the resource comprising a contenthaving a content data convertible into a text data and one or moreresource property fields defining at least one characteristic of theresource; sectioning the content data into one or more content portionsbased on an analysis of at least one of the content data and the one ormore resource property fields; and assigning at least one contentportion of the one or more content portions to at least one learningobjective.
 2. The method of claim 1, wherein sectioning the content datainto the one or more content portions based on the analysis of the atleast one of the content data and the one or more resource propertyfields comprises: determining whether the one or more resource propertyfields includes one or more resource structure fields, the one or moreresource structure fields defining a content structure of the contentdata and the content structure including one or more data hierarchylevels; and in response to determining the one or more resource propertyfields includes the one or more resource structure fields, sectioningthe content data into the one or more content portions according to atleast one data hierarchy level of the one or more data hierarchy levels.3. The method of claim 1, wherein assigning the at least one contentportion to the at least one learning objective comprises: applying asemantic analysis to the at least one content portion and applying thesemantic analysis to each learning objective of the one or more learningobjectives; based on results of the semantic analysis, assigning arelevance score for the at least one content portion in respect of atleast one learning objective of the one or more learning objectives, therelevance score representing an estimated degree of correlation betweenthe at least one content portion and the at least one learningobjective; for each learning objective, determining whether therespective relevance score assigned to the at least one content portionat least satisfies a relevance threshold for that learning objective,the relevance threshold being a minimum relevance score required for theat least one content portion to be associated with that learningobjective; and in response to determining the relevance score at leastsatisfies the relevance threshold, assigning the at least one contentportion with that learning objective.
 4. The method of claim 1, whereinthe one or more content portions comprises two or more data hierarchylevels.
 5. The method of claim 1, wherein the resource comprises avideo, and receiving the resource for satisfying the at least onelearning objective comprises transcribing an audio data into the textdata.
 6. The method of claim 1, wherein the resource comprises an image,and receiving the resource for satisfying the at least one learningobjective comprises applying an electronic character recognitionconversion to the image for generating the text data from the image. 7.An electronic learning system comprising: a memory for storing one ormore learning objectives; and a processor in electronic communicationwith the memory, the processor operating to: receive a resource forsatisfying at least one learning objective of the one or more learningobjectives, the resource comprising a content having a content dataconvertible into a text data and one or more resource property fieldsdefining at least one characteristic of the resource; section thecontent data into one or more content portions based on an analysis ofat least one of the content data and the one or more resource propertyfields; and assign at least one content portion of the one or morecontent portions to at least one learning objective.
 8. The electroniclearning system of claim 7, wherein the processor operates to: determinewhether the one or more resource property fields includes one or moreresource structure fields, the one or more resource structure fieldsdefining a content structure of the content data and the contentstructure including one or more data hierarchy levels; and in responseto determining the one or more resource property fields includes the oneor more resource structure fields, section the content data into the oneor more content portions according to at least one data hierarchy levelof the one or more data hierarchy levels.
 9. The electronic learningsystem of claim 7, wherein the processor operates to: apply a semanticanalysis to the at least one content portion and apply the semanticanalysis to each learning objective of the one or more learningobjectives; based on results of the semantic analysis, assign arelevance score for the at least one content portion in respect of atleast one learning objective of the one or more learning objectives, therelevance score representing an estimated degree of correlation betweenthe at least one content portion and the at least one learningobjective; for each learning objective, determine whether the respectiverelevance score assigned to the at least one content portion at leastsatisfies a relevance threshold for that learning objective, therelevance threshold being a minimum relevance score required for the atleast one content portion to be associated with that learning objective;and in response to determining the relevance score at least satisfiesthe relevance threshold, assign the at least one content portion withthat learning objective.
 10. The electronic learning system of claim 7,wherein the one or more content portions comprises two or more datahierarchy levels.
 11. The electronic learning system of claim 7, whereinthe resource comprises a video, and the processor operates to transcribean audio data into the text data.
 12. The electronic learning system ofclaim 7, wherein the resource comprises an image, and the processoroperates to apply an electronic character recognition conversion to theimage for generating the text data from the image.